Method of executing test scripts against multiple systems

ABSTRACT

A method of executing a plurality of tests scripts or one or more test script sets against a plurality of systems to be tested. The method comprises providing an instance of a test script set on a server, writing an initial system value corresponding to an initial system to be tested on a web page file hosted on the server, retrieving the initial system value from the web page file and launching a URL using the initial system value retrieved to execute the test script set against the initial system to be tested. The method further includes overwriting the web page file to change the initial system value to a next system value corresponding to a next system to be tested, retrieving the next system value from the web page file and launching a next URL using the next system value retrieved to execute the test script set against the next system to be tested.

FIELD OF THE INVENTION

The present invention relates to a method of executing one or more testscripts or one or more test script sets against a plurality of computersystems to be tested.

BACKGROUND OF THE INVENTION

System integration testing often involves the use of regression testing.Regression testing is the process of testing changes to computerprograms to make sure that the older programming still works with thenew changes. In large organizations, regression testing is a part of thesoftware development process and is carried out by softwareprofessionals or testing specialists. These software professionalsdevelop test sets or test script sets that will test new units of code.All or a subset of the established test sets or test script sets may bereused. When a new version of software is installed, the old test scriptsets are run against the new version to make sure that all the oldcapabilities of the software still work. Often organizations need totest the implementation of a software program across multiple computersystems or geographies. Thus, there is a need for a system and method ofexecuting test scripts across multiple computer systems or geographiesin a more efficient and economic manner by an organization.

SUMMARY OF THE INVENTION

In a first aspect of the invention, there is provided a method ofexecuting a set of automated test scripts against a plurality ofsystems. The method comprises the steps of providing an instance of theset of automated test scripts on a server, writing to asystem-under-test web page file hosted on the server an initialsystem-under-test value corresponding to an initial system to be tested,retrieving the initial system-under-test value from thesystem-under-test web page file and launching an initial URL using theinitial system-under-test value retrieved to execute the set ofautomated test scripts against the initial system to be tested. Thewriting step preferably comprises creating a set-system-under-testscript for each of the plurality of systems and executing an initial oneof the set-system-under-test script created to generate thesystem-under-test web page file having the initial system-under-testvalue. The retrieving step preferably comprises creating aget-system-under-test script for navigating to the system-under-test webpage file on the server, adding the get-system script to the beginningof each one of a plurality of test scripts within the set of automatedtest scripts and executing the get-system-under-test script to retrievethe initial system-under-test value from the system-under-test web pagefile. The launching step preferably comprises converting the initialsystem-under-test value to the initial URL and navigating a browser tothe initial URL to execute the set of automated test scripts. The methodpreferably includes the steps of overwriting the system-under-test webpage file to change the initial system-under-test value to a nextsystem-under-test value corresponding to a next system to be tested,retrieving the next system-under-test value from the system-under-testweb page file and launching a next URL using the next system-under-testvalue to execute the set of automated test scripts against the nextsystem to be tested. The overwriting step preferably comprises executinganother one of the set-system-under-test script created to change theinitial system-under-test value to the next system-under-test value onthe web page file. The method preferably comprises the steps ofrepeating the overwriting, retrieving and launching steps until the setof automated test scripts have been executed against each of theplurality of systems to be tested. In a preferred embodiment, the set ofautomated test scripts are created using a test script execution engine,whereas, the set of automated test scripts are executed, preferably,using a test script execution manager. Furthermore, in a preferredembodiment, the web page file is a static web page file.

In another embodiment of the invention, there is provided a method ofexecuting multiple sets of automated test scripts against a plurality ofsystems to be tested. The method comprises the steps of providing a testscript execution manager for executing the multiple sets of automatedtest scripts, and providing an instance of the multiple sets ofautomated test scripts, preferably, on a server. The method preferablycomprises appending a get-system script to the beginning of eachindividual test script within each of the multiple sets of automatedtest scripts for retrieving a desired system value corresponding to adesired one of the plurality of systems to be tested, selecting adesired one of a plurality of set-system scripts corresponding to thedesired one of the plurality of systems to be tested, and selecting oneor more test script sets from the multiple sets of automated testscripts to be executed against the desired one of the plurality ofsystems. The appending step preferably comprises creating a get-systemscript for retrieving from a web page file the desired system valuecorresponding to the desired one of the plurality of systems to betested. The step of selecting a desired one of a plurality of set-systemscripts preferably comprises creating a plurality of set-system scriptsfor writing a respective system value to the web page file hosted on theserver, each of the plurality of set-system scripts corresponding to arespective one of the plurality of systems to be tested, and adding thedesired one of the plurality of set-system scripts to the beginning ofthe one or more test script sets selected, such that each of the one ormore test script sets selected is executed against the desired one ofthe plurality of systems to be tested. Moreover, the method comprisesrepeating both the selecting steps until at least one set from themultiple sets of automated test scripts has been selected for each ofthe plurality of systems to be tested. Further, the method preferablycomprises directing the test script execution manager to execute the oneor more test script sets selected against a respective one of theplurality of systems to be tested selected. The directing steppreferably comprises executing the desired one of the plurality ofset-system scripts for setting the desired system value corresponding tothe desired one of the plurality of systems to be tested, executing eachof the one or more test script sets selected against the desired one ofthe plurality of systems to be tested, such that the get-system scriptis executed each time and before the execution of each individual testscript within each of the one or more test script sets selected, andrepeating both the executing steps until each of the multiple sets ofautomated test scripts selected have been executed against the selectedplurality of systems to be tested. The step of executing each of the oneor more test script sets preferably comprises navigating to the web pagefile on the server, reading the desired system value from the web pagefile, and launching an appropriate URL corresponding to the desiredsystem value to execute each of the one or more test script setsselected against the desired one of the systems to be tested.

In yet another embodiment of the invention, there is provided aprogrammable media containing programmable software to execute a set ofautomated test scripts against a plurality of systems to be tested, theprogrammable software comprising the steps of accessing on a server aninstance of the set of automated test scripts, writing to a web pagefile a system value corresponding to a desired one of the plurality ofsystems to be tested, retrieving the system value from the web pagefile, and launching a URL using the system value to execute the set ofautomated test scripts against the desired one of the plurality ofsystems to be tested. The writing step preferably comprises creatingindividual set-system scripts for each of the plurality of systems to betested, adding an initial one of the set-system scripts to the beginningof the set of automated test scripts, and executing the initial one ofthe set-system scripts to generate the web page file having the systemvalue corresponding to the desired one of the plurality of systems to betested. The retrieving step preferably comprises creating a get-systemscript for retrieving from the web page file the system valuecorresponding to the desired one of the plurality of systems to betested, and adding the get-system script to the beginning of each one ofa plurality of test scripts within the set of automated test scripts,and executing the get-system script to retrieve the system value fromthe web page file. The launching step preferably comprises convertingthe system value to the URL, and navigating a browser to the URL toexecute the set of automated test scripts. The programmable mediasoftware preferably comprises overwriting the web page file to changethe system value to a next system value corresponding to a next desiredone of the plurality of systems to be tested, retrieving the next systemvalue from the web page file, launching a next URL using the next systemvalue to execute the set of automated test scripts against the nextsystem to be tested, and repeating the overwriting, retrieving andlaunching steps until the set of automated test scripts have beenexecuted against each of the plurality of systems to be tested. Theoverwriting step preferably comprises executing another one of theset-system scripts to change the system value to the next system valueon the web page file. In a preferred embodiment, the web page file ishosted on the same server as the test scripts.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings which are incorporated in and form a part ofthis specification, illustrate embodiments of the invention and,together with the description, serve to explain the principles of theinvention.

FIG. 1 is a flowchart demonstrating a method of executing a set of testscripts against a plurality of systems to be tested, in accordance withan embodiment of the invention.

FIG. 2 is a diagram demonstrating an execution sequence of a set of testscripts against a plurality of systems to be tested, in accordance withan embodiment of the invention.

FIG. 3 is a table identifying the different scripts that are used forexecuting test script sets against multiple systems to be tested, inaccordance with an embodiment of the invention.

FIG. 4 illustrates a method of executing a set of automated test scriptsagainst a plurality of systems to be tested, in accordance with anembodiment of the invention.

FIG. 5 illustrates a method of executing multiple sets of automated testscripts against a plurality of systems to be tested, in accordance withan embodiment of the invention.

FIG. 6 shows an example of a source code listing for generating a systemfile having a system value corresponding to a system to be tested, inaccordance with an embodiment of the invention.

FIG. 7 shows an example of a source code listing for retrieving a systemvalue from a system file corresponding to a system to be tested, inaccordance with an embodiment of the invention.

BEST MODE FOR CARRYING OUT THE INVENTION

It will be apparent to those skilled in the art that variousmodifications and variations can be made to the present inventionwithout departing from the spirit and scope of the invention. Thus, itis intended that the present invention cover the modifications andvariations of this invention provided they come within the scope of theappended claims and their equivalents. Reference will now be made indetail to the preferred embodiments of the invention.

In one embodiment, the invention provides a method of executing a testset or a test script set (which is made up of a plurality of individualtest scripts) for testing software installed on multiple computersystems. The method comprises providing an instance of a test script seton some type of a storage media, such as a disk or hard drive,preferably, a network drive, and more preferably, on a web server. Asused herein, a test script set or test set refers to a set of automatedand/or regression test scripts that have been created, preferably, usinga commercially available test script execution tool or software.Preferably, each of these automated test scripts are grouped in a testscript set, where the test script set is designed to achieve a specifictesting goal, for example, to test a new version of an application or totest a specific function in an application. When creating a test scriptset, preferably, a selection is made as to which of the automated testscripts are to be included in that set. However, in a preferredalternative embodiment, instead of a test script set, a plurality oftest scripts (one or more test scripts) are executed against multiplesystems. In order to direct an instance of a test script set or testscripts to be executed against multiple systems, separate utilityscripts called “set-system-under-test” or “set-system” scripts arecreated for each one of the computer systems to be tested. Eachset-system script generates a data file, more preferably, a web page orweb page file (also, referred to as system web page file orsystem-under-test web page file) that has written on it data or morespecifically, a “system value” or a “system-under-test value” thatcorresponds to a computer system to be tested (that is, one system outof the plurality of systems to be tested). In a preferred embodiment,the web page file is a static web page file, however, the web page filecreated can be a dynamic web page that holds data specifying whichsystem is to be tested. As used herein, the “system value” represents amnemonic or code that corresponds to a specific computer system.Further, each computer system is represented by a unique UniformResource Locator (URL).

In order to run any of the test scripts or test script sets against aparticular computer system, a test script first has to navigate to theURL of that computer system. As such, a second utility script orsubroutine called “get-system-under-test” or “get-system” script iscreated for retrieving the system value corresponding to the computersystem to be tested, which has been written by the set-system scriptonto the web page file. In a preferred embodiment, the get-system scriptis a subroutine that is added to each of the test scripts within a testscript set. It is the get-system script that enables retrieval of thesystem value corresponding to a system to be tested. In particular, theget-system script navigates to the web page file and reads in the dataor system value set by the set-system script. In a preferred embodiment,the system value written in the web page file is a mnemonic or code,which is looked up in a table that contains the corresponding URL(Uniform Resource Locator). Preferably, the system value is convertedinto the corresponding URL, a browser is navigated to that URL, suchthat, the URL corresponding to the system value is launched and the testscript set is executed against that system. Moreover, if the test scriptset is to be executed against another computer system, then theappropriate set-system script is executed to overwrite the web pagefile, such that the initial system value is replaced with a new or nextsystem value corresponding to that another computer system. Further,each of the test scripts within the test script set is seriallyexecuted, such that the get-system subroutine is executed first toretrieve the new or next system value for launching a next URLcorresponding to that another system, followed by the execution of theactual test script. In a preferred embodiment, the set-system scriptsare created using a commercially available test script execution engineor tool, more preferably, using the same engine or tool used forcreating the test scripts. Also, the get-system script is created,preferably, using a commercially available test script execution engineor tool, more preferably, using the same engine or tool used forcreating the test scripts. In a preferred embodiment, the test scripts,the set-system scripts and the get-system script are created usingMercury's QuickTest® Professional test script execution engine or tool.

The process of executing a set of automated test scripts against aplurality of systems is depicted in the flow chart 100 shown in FIG. 1.As shown in FIG. 1, there are n number of systems to be tested and thereare m number of test scripts to be run on each of the n systems. Themethod shown in FIG. 1, includes the creation of a utility script called“setSystemUnderTest_(i)”, which writes to a web page file a system valuerepresenting a desired or current system under test against which thetest scripts are to be executed. The method further includes creating asecond utility script or a subroutine called “getSystemUnderTest” (notshown explicitly in FIG. 1), which is added to the beginning of each ofthe plurality of test scripts m, so that when “run_script_(j) isexecuted in step 120, the getSystemUnderTest subroutine is executedfirst before the actual test script. Further, “i” is the index valuerepresenting a different system and “j” is the index value representinga different automated test script. As shown in the flowchart in FIG. 1,both the index values for i and j are initially set to 0 in step 110,such that the first test script called is script_0 and the first systemtested is system_0. The utility script “setSystemUnderTest_(i)” isexecuted at the start in step 112 to set or write in step 114 the firstdesired or current system under test value on the web page file called“systemUnderTest.htm”, denoted by numeral 116. At the start of executionof each of the automated test scripts “run_script_(j)” in step 120,first, the subroutine “getSystemUnderTest” appended to the beginning ofeach of the test scripts is called to retrieve or read the desired orcurrent system under test value in step 118, which has already been setin step 114 on the web page file “systemUnderTest.htm” 116. Preferably,it is only after the system value has been retrieved that the actualscript is executed in step 120. Next, as shown in step 122, the indexvalue j is compared to m (the total number of test scripts) to seewhether there are any more test scripts that need to be executed againstthe first system. If j<m, then the test script “run_script_(j) isexecuted again, as shown by the loop 124, but this time the index valueis incremented by one (as depicted by “j++” in step 120), so that thenext test script is executed. However, since the subroutine“getSystemUnderTest” is appended to each of the test scripts, the“getSystemUnderTest” script is executed first and the script navigatesto the systemUnderTest.htm web page 116 and reads in the system value,step 118, contained on the web page 116 and then the actual script isexecuted in step 120. Again, in Step 122, the index value j is comparedto m (the total number of test scripts) to see whether there are anymore test scripts that need to be executed against the first system. Aslong as j<m, the loop 124 is repeated to include steps 118 through 122.When j is not less than m, the index value for i is compared to n (thetotal number of systems to be tested) in step 126. If i<n in step 126,then loop 128 is carried out such that, the “setSystemUnderTest_(i)” isexecuted again, but this time the index value i is incremented by one(as depicted by “i++” in step 112), to overwrite the system value (thus,representing the next system to be tested) in step 114 on the web pagesystemUnderTest.htm 116 before executing the script in step 120. Again,each of the scripts is executed against the new system, since the newsystem value written on the web page file 116 is read in step 118 priorto executing each of the test scripts in step 120. The system value isnot changed until each of the scripts has been executed against thesystem set in the web page file. Thus, when j is not less than m, theindex value for i is compared to n (the total number of systems to betested) in step 126, the loop 128 is carried out such that, the“setSystemUnderTest_(i)” is executed again, but this time the indexvalue is incremented by one (as depicted by “i++”), thus, writing in anew system value (representing yet another system to be tested) in step114 on the web page systemUnderTest.htm 116 before executing each of thetest scripts in step 120. This process continues until each of the testscripts m has been executed against each of the n plurality of systemsto be tested.

In yet another embodiment of the invention, the invention provides amethod of executing multiple sets of automated test scripts against aplurality of systems to be tested, preferably, using a commerciallyavailable test script execution manager for executing the multiple setsof automated test scripts. The automated test scripts are created,preferably, using a commercially available test script execution engineor tool and are stored, preferably, on a web server for wider access,more preferably, on the same web server as the test script executionmanager, such that, software installed on the plurality of computersystems can be tested without requiring any manual intervention forchanging execution of a test script set from a desired computer systemto a different or next desired system. A plurality of set-system scriptsare created for writing a respective system value to a web page filehosted on the web server, such that each of the plurality of systems hasa respective set-system script. The set-system scripts are created,preferably, using a commercially available test script execution engineor tool, more preferably, using the same engine or tool used forcreating the test scripts, such as, Mercury's QuickTest® Professionaltest script execution engine or tool. In a preferred embodiment, the webpage file is a static web page file, however, the web page file createdcan be a dynamic web page that holds data specifying which system is tobe tested. Further, a get-system script is appended to the beginning ofeach of the individual test script within each of the multiple sets ofautomated test scripts, such that the get-system script is executedfirst to retrieve from a web page file a desired system valuecorresponding to a desired one of the plurality of systems to be tested.The get-system script is created, preferably, using a commerciallyavailable test script execution engine or tool, more preferably, usingthe same engine or tool used for creating the test scripts, such asMercury's QuickTest® Professional test script execution engine or tool.When using a test execution manager, one selects and adds a desired oneof a plurality of set-system scripts corresponding to the desired one ofthe plurality of systems to be tested to an execution sequence and,further, selects and adds one or more test script sets from the multiplesets of automated test scripts to be executed against the desired one ofthe plurality of systems selected. The selection and addition of anotherdesired system to be tested and the selection and addition of testscript sets to be executed are repeated until at least one set from themultiple sets of automated test scripts has been selected for each ofthe plurality of systems to be tested. The test script execution managerexecutes the one or more test script sets selected against the selectedsystems in the order that they are listed within the test scriptexecution manager. In particular, since a set-system script is firstselected and added to the execution sequence within the test scriptexecution manager, the set-system script is executed first to set thedesired system value corresponding to the desired one of the pluralityof systems to be tested. Then each of the one or more test script setsselected and added to the execution sequence is executed against thedesired one of the plurality of systems to be tested, until all thedesired test script sets added to the execution sequence have beenindividually executed against the desired system. Further, theget-system script is executed each time and before the execution of eachindividual test script within each of the one or more test script setsin the execution sequence. In particular, the get-system subroutine orscript navigates to the web page file on the server, reads the desiredsystem value from the web page file, and launches an appropriate URLcorresponding to the desired system, thus, ensuring that the test scriptis being executed against the desired system.

Referring to FIG. 2, numeral 200 depicts a test execution sequence forexecuting a test script set against a plurality of systems. As shown inFIG. 2, there are n number of systems to be tested and there are mnumber of test scripts to be run on each of the n systems. A desired oneof the set-system utility scripts is first selected or added to a testexecution sequence, preferably, to a test execution sequence in acommercially available test script execution manager or tool, so thatthe execution of the test scripts can be carried out automaticallywithout any manual intervention. Then, one selects or adds a test scriptset that is to be executed against the system that has been selected.Next, another desired one of the set-system-under-test scripts isselected or added to the test execution sequence. Again, another testscript set is added to the test execution sequence. This process isrepeated until each of the systems to be tested as well as the testscript set to be executed is added to the test execution sequence. Thelisting in the test execution sequence dictates the order in which thetests will be carried out, that is, which system will be tested firstand what test scripts or test script sets will be executed against thatsystem. Accordingly, FIG. 2 depicts a sequential order in which anexecution sequence 200 is carried out. The execution sequence 200 is asfollows: “setSystemUnderTest_(—)0”, “run-script_(—)0”, “run_script_(—)1”. . . “run_script_(m-1)”, “setSystemUnderTest_(—)1”, “run_script_(—)0”,“run_script_(—)1” . . . “run_script_(m-1), “setSystemUnderTest_(n-1)”,“run_script_(—)0”, “run_script_(—)1” . . . “run_script_(m-1), insequence. In the execution sequence 200, setSystemUnderTest_(—)0 isexecuted first (in step 210) to set the system value to “System 0”, thefirst system that is going to be tested, and script_(—)0 is executednext (in step 212), followed by the execution of script_(—)1 (in step214) through (shown by “ . . . ” in step 216) script_(m-1), in step 218,are all executed in consecutive order against System 0 as set forth inthe execution sequence 200. Similarly, the setSystemUnderTest_(—)1 instep 220 signifies that the setSystemUnderTest_(—)1 script is executednext to overwrite the current system value to a new system value,namely, changing the system from System 0 to System 1. Thus, in steps222 to steps 228, script_(—)0, script_(—)1 through script_(m-1) are allexecuted in consecutive order against System 1. Furthermore, the “ . . .” in step 230 signifies that additional systems and scripts to beexecuted against those systems can be added to the execution sequence asdesired. Finally, the setSystemUnderTest_(n-1) in step 140 signifiesthat the setSystemUnderTest_(n-1) is executed to overwrite the currentsystem value on the web page file to a new system value corresponding toSystem (n-1) and that in steps 242 to 148, script_(—)0 throughscript_m-1 are all going to be executed in consecutive order againstSystem (n-1). Thus, each of the scripts is executed against each of thesystems, as selected in the execution sequence 200.

FIG. 3 depicts a table 300 outlining the various scripts and theircorresponding function. In table 300, 310 is the“setSystemUnderTest_(i)” utility script that updates the current systemunder test by replacing the existing systemUnderTest.htm file with HTMLcode selecting the new system under test value. Given a set of m systemsto be tested, this script will execute for i={0, . . . , (m-1)}, whereeach index value represents a different target test system. Next, 314 isthe getSystemUnderTest script or subroutine that navigates to thesystemUnderTest.htm web page, reads the value of the current systemunder test and saves it in a variable called sysUnderTest_value_out. Thescript uses this value to navigate to the URL of the system to be testedand executes. Given a set of n automated scripts, run_script_(j) willrun on each system under test for j={0, . . . , (n-1)}, where each indexvalue represents a different automated script. Similarly, 318 representsthe convertSysToURL( ) script, which is a generic algorithm whose inputis a mnemonic or some code associated with a system under test and whoseoutput is its corresponding Internet URL. The setBrowserTo( ) 322 is ageneric algorithm whose input is an Internet URL and whose output is theact of navigating an Internet browser to that URL. Finally, in 326,run_script_(j) represents the execution of an automated script. Eachautomated script starts by first executing the subroutinegetSystemUnderTest, numeral 314, which retrieves the system valuewritten on the web page file, systemUnderTest.htm. Then, the scriptconvertSysToURL( ), numeral 318, is executed to convert the retrievedsystem value to a corresponding Internet URL. Finally, the scriptsetBrowserTo( ), numeral 322, is executed to navigate an Internetbrowser to the corresponding Internet URL so that the script can beexecuted.

Referring to FIG. 4, FIG. 4 demonstrates an example of an executionsequence 400 for executing a set of automated test scripts (called “Set1”) against three systems to be tested (called “System 1”, “System 2”and “System 3”, respectively). In particular, a “Set-System” utilityscript called “Set-System 1 Script” is first added to the executionsequence in step 410 followed by the addition of test script set, “Set1” in step 412. Then, a “Set-System 2 Script” utility script is added tothe execution sequence in step 430 followed by the addition of testscript “Set 1” in step 432 and, finally, a utility script “Set-System 3Script” is added in step 450, followed by the addition of test script“Set 1” in step 452. Also, as shown in FIG. 4, “Set 1” to be executedagainst each of the three systems 1, 2 and 3, in steps 412, 432 and 452,respectively, is made up of three test scripts, namely, “Test Script 1”,“Test Script 2” and “Test Script 3”. Furthermore, a utility script orsubroutine “Get-System Script” is added to the beginning of each of thetest scripts within a test script set. In particular, as shown in FIG.4, the “Get-System Script” is added to the beginning of each of thethree test scripts “Test Script 1”, “Test Script 2” and “Test Script 3”.The “Set-System 1 Script” is executed first in step 410, such that a webpage file is generated (not shown in FIG. 4). The web page file haswritten in it the system value (or system-under-test value)corresponding to the first system to be tested, namely, System 1. Next,each of the test scripts Test Script 1, Test Script 2 and Test Script 3within Set 1 is executed in step 412. In particular, when Test Script 1is executed, the Get-System subroutine appended to the beginning of TestScript 1 is executed first, which means that the Get-System Script(subroutine) navigates a browser, such as, Microsoft's Internet Explorerto the generated web page file and retrieves the system value written onthe web page file by the Set-System 1 Script in step 410. The browserreads the system value and launches a URL corresponding to the systemvalue for System 1 so that Test Script 1 is executed against System 1.Next, the test management tool executes the second test script, TestScript 2, such that, again the Get-System subroutine appended to thebeginning of Test Script 2 is executed first and reads in the systemvalue corresponding to System 1 from the web page file, thus, directingexecution of Test Script 2 against System 1. Similarly, Test Script 3 isexecuted against System 1, since the system-under-test web page filestill has written in it the system value for System 1.

Next, the utility script “Set-System 2 Script” is executed in step 430,which overwrites the system value in the web page file, such that, thesystem value now corresponds to “System 2”. Thus, when Set 1 is executedin step 432, the “Get-System Script” attached to the beginning of “TestScript 1” is executed first, which means that the Get-System script orsubroutine navigates to the overwritten web page file and retrieves thesystem value corresponding to System 2 from the web page file. Inparticular, the browser reads the system value corresponding to System 2and launches a URL corresponding to System 2, so that Test Script 1 isexecuted against System 2. Next, Test Script 2 is executed, such that,again the Get-System Script appended to the beginning of Test Script 2is executed first, ensuring that Test Script 2 is executed againstSystem 2 as specified by the system value written on the web page file.Similarly, Test Script 3 is executed against System 2. Next, the“Set-System 3 Script” is executed in step 450. The Set-System 3 utilityscript overwrites the system value on the web page file to now set thesystem value to that corresponding to System 3. Again, each of the testscripts within Set 1 is executed in step 452, with the Get-System Scriptbeing executed first, such that, the Get-System Script navigates thebrowser to the overwritten web page file and retrieves the overwrittensystem value corresponding to System 3 from the web page file each timeand before the execution of each of the respective Test Scripts 1, 2 and3. Thus, when Set 1 is executed in step 452, Test Script 1, Test Script2 and Test Script 3 are all executed against System 3. Accordingly, inthis manner, if Set 1 has to be executed against more systems, then onecan add additional systems one at a time, each system followed by theaddition of Set 1. This process can be repeated until all the additionalsystems to be tested are listed in the execution sequence, so that thetest management tool can carry out the execution of the scripts in asequential order as explained herein above.

Preferably, both the set-system and get-system scripts are created usinga commercially available functional enterprise testing tool, such asMercury's QuickTest® Professional software, whereas, the executionsequence is carried out using another commercially available testmanagement tool, namely, Mercury's Test Director® software. Moreover, ina preferred embodiment, the system-under-test value corresponds to a URL(Uniform Resource Locator), which is looked up in a table, so that, theURL corresponding to a system to be tested is launched and the testscript set is executed against that system. Further, in a preferredembodiment, the web page file that is generated is a web page file thatis hosted on a server, more preferably, on the same server as the testscript set or sets. Alternatively, the web page file can be stored on adisk, a hard drive of a computer or on a network drive. Further yet,instead of the utility script Set-System writing in a system value or amnemonic corresponding to a system to be tested on a web page file, thescript can be designed to write in the actual URL for a system to betested, thus, skipping the step of having to look up the correspondingURL for a system value in a table. Moreover, although the invention hasbeen described as creating a static web page file, alternatively, theweb page file created can be a dynamic web page that holds dataspecifying which system is to be tested. In a dynamic web pageimplementation, preferably, an execution environment, such as PERL[Practical Extraction and Report Language] would be installed on a webserver. Accordingly, the Set-System script would no longer need toperform file operations to update the web page file,systemUnderTest.htm. Instead, the Set-System script would navigate tothe systemUnderTest.htm web page, select the desired system from adrop-down list of system values, and, preferably, click a “submit”button. Upon clicking “submit”, a script on the web server, preferably,written using PERL, would update the web page file, systemUnderTest.htm,to reflect the system value chosen by the Set-System script.

FIG. 5 shows another example of an execution sequence 500, where one ormore test script sets are scheduled to be executed against each of threeseparate computer systems, in accordance with an embodiment of theinvention. The three (3) different computer systems to be tested aredenoted as “System 1”, “System 2” and “System 3”. Further, the testscripts chosen to be executed against each of the three systems arechosen from the following three test sets, namely, “Set 1”, “Set 2” and“Set 3”. In particular, Set 1, Set 2 and Set 3 are all chosen to beexecuted against System 1 (as shown in steps 512, 514 and 516), whereas,only Set 1 and Set 2 are selected to be executed against System 2 (asshown in steps 522 and 524), and, finally, test script sets, Set 2 andSet 3 (as shown in steps 532 and 534) are selected to be executedagainst System 3. Also, there are three separate utility scripts thatare created to correspond to each of the three systems to be tested. Inparticular, “Set-System 1 Script” corresponds to System 1, “Set-System 2Script” corresponds to System 2, and “Set-System 3 Script” correspondsto System 3. In order to execute one or more of the three test scriptsets against each of the three systems, a user has to set forth anexecution sequence, that is, the sequential order in which the scriptswill be executed, preferably, using a commercially available testingtool. In the example, “Set-System 1 Script” is first selected to beadded to the execution sequence in 510. Then, the test script set “Set1” is added to the execution sequence in 512, followed by “Set 2” instep 514 and “Set 3” in step 516, respectively. Similarly, “Set-System 2Script” is then selected to be added to the execution sequence in step520. Then, the test script sets “Set 1” and “Set 2” in steps 522 and524, are all added, respectively. Finally, “Set-System 3 Script” isselected to be added to the execution sequence in step 530. Then, thetest script sets “Set 2” and “Set 3” are all added, in steps 532 and534, respectively. As shown in FIG. 5, “Set 1” is made up of “TestScript 1” and Test Script 2”, “Set 2” is made up of “Test Script 3”,“Test Script 4” and “Test Script 5”, whereas, “Set 3” is made up of“Test Script 6” and “Test Script 7”. As previously explained, thegrouping of the test scripts within a test script set can be customizedin any number of ways and in accordance with one or more testingobjectives.

Furthermore, as shown in FIG. 5, a subroutine or utility script called“Get-System” is added to the beginning of each of the test scriptswithin each test set, that is, the Get-System subroutine is added to thebeginning of each of the Test Scripts 1 through 7. Accordingly, duringthe execution sequence carried out for the example in FIG. 5, thescripts are executed in order, as follows: Set-System 1 Script,Get-System Script, Test Script 1, Get-System Script, Test Script 2,Get-System Script, Test Script 3, Get-System Script, Test Script 4,Get-System Script, Test Script 5, Get-System Script, Test Script 6,Get-System Script, Test Script 7, Set-System 2 Script, Get-SystemScript, Test Script 1, Get-System Script, Test Script 2, Get-SystemScript, Test Script 3, Get-System Script, Test Script 4, Get-SystemScript, Test Script 5, Set-System 3 Script, Get-System Script, TestScript 3, Get-System Script, Test Script 4, Get-System Script, TestScript 5, Get-System Script, Test Script 6, Get-System Script, and TestScript 7. Thus, each test script is independent and contains thesubroutine Get-System, which gets the system value from the web page forits own use and runs against the URL corresponding to the system itfinds on the web page. In this manner, if one wants to execute testscript sets against additional systems, the additional systems can beadded to the execution sequence as outlined above, so that manualintervention is not required to execute test sets against multiplesystems.

FIG. 6 provides an example of a source code for the “Set-System” or“SetSystemUnderTest” utility script. The set-system utility firstprepares an HTML file for writing, then writes to the HTML file andcloses the HTML file. As shown in FIG. 6, the set-system script writesin a system value corresponding to the first system to be tested,namely, System 1. However, the system value can be changed from System 1to any one of the other three systems, namely, System 2, System 3 orSystem 4. As shown in FIG. 6, the set-system script chooses a system byadding the word “selected” to the HTML <option> tag associated with thefirst system to be tested, namely, System 1. The tag then reads:“<option selected>System 1</option>. Similarly, the system value can bechanged from System 1 to any one of the other three systems, namely,System 2, System 3 or System 4 by removing the word “selected” from the<option> tag associated with System 1 and adding it to the <option> tagassociated with any one of the other three systems.

FIG. 7 provides an example of a source code for the utility script orsubroutine “Get-System” or “GetSystemUnderTest”. As shown, the systemvalue corresponding to a system to be tested is stored in a variablecalled “sysUnderTest_value_out”, which is the value that is returned tothe calling automated test script. This value is then used to navigatethe browser to the correct URL and to run the automated test scriptagainst the URL of the system to be tested.

Furthermore, in yet another embodiment, the invention provides a systemthat comprises a computer terminal or an equivalent device, a computerreadable storage media or programmable media containing programmablesoftware to execute one or more test script sets against multiple or aplurality of systems in a form that is executable by a computer system.The programmable software comprises the steps of accessing on a serveran instance of a set of automated test scripts, writing to a web pagefile or web page file a system value corresponding to a desired one ofthe plurality of systems to be tested, the web page file being hosted,preferably, on the same server as the set of test scripts. The writingfurther comprises creating individual set-system scripts for each of theplurality of systems to be tested, adding an initial one of theset-system scripts to the beginning of the set of automated testscripts, and executing the initial one of the set-system scripts togenerate the web page file having written on it the system valuecorresponding to the desired one of the plurality of systems to betested. The programmable software further comprises retrieving thesystem value from the web page file. The retrieval comprises creating aget-system script for retrieving from the web page file the system valuecorresponding to the desired one of the plurality of systems to betested, adding the get-system script to the beginning of each one of aplurality of test scripts within the set of automated test scripts andexecuting the get-system script to retrieve the system value from theweb page file. The software further comprises launching a URL using thesystem value to execute the set of automated test scripts against thedesired one of the plurality of systems to be tested. The launchingfurther comprises converting the system value to the URL and navigatinga browser to the URL to execute the set of automated test scripts.Moreover, to execute the set of automated test scripts against theplurality of systems, the programmable software comprises overwritingthe web page file to change the system value to a next system valuecorresponding to a next desired one of the plurality of systems to betested by executing another one of the set-system scripts to change thesystem value to the next system value on the web page file. The softwareincludes retrieving the next system value from the web page file andlaunching a next URL using the next system value to execute the set ofautomated test scripts against the next system to be tested. The stepsof overwriting the web page file to set a new system value, retrievingthat new system value and launching an appropriate URL corresponding tothat new system value are repeated until the set of automated testscripts have been executed against each of the plurality of systems tobe tested. After an automated test script set has been executed, thesoftware provides a pass/fail execution report that can be analyzed by asoftware professional. Furthermore, the pass/fail results or report ofthe automated tests may be displayed on a visual display such as acomputer monitor screen, printed out or stored in a memory eitherinternal to the terminal or on a disc for later use.

In a preferred embodiment, a software professional or user uses acomputer terminal or an equivalent device for accessing the one or moresets of automated test scripts stored, preferably, on a server oralternatively, stored in a memory either internal to the computerterminal or external to the computer terminal. Similarly, the tests maybe performed either on the same computer terminal or on a web serverlinked to an electronic network, or on a client PC linked to anelectronic network. Further, in a preferred embodiment, the test scriptsare created using Mercury's QuickTest® Professional test scriptexecution engine or tool, whereas, the execution of the test scripts onthe various systems are carried out by means of a commercially availabletest script execution manager or management tool, such as, Mercury'sTestDirctor® software program. Preferably, the TestDirectore® softwareis installed on an organization's web server, along with the automatedtest scripts created using Mercury's QuickTest® Professional software.Moreover, preferably, the web page file is hosted on the same server asthe TestDirector® software. Mercury's QuickTest® Professional softwareis installed on individual computer systems or workstations, such that,different individuals that are part of a software testing team cancreate one or more test scripts for testing a specific aspect or allaspects of an application or web site. Further, these tests can be runevery time a web site or application changes. Additionally, after a testscript set has been executed against one or more systems, Mercury'sTestDirector® software provides an online pass/fail execution reportthat can be analyzed by software professionals within the organization.

The foregoing descriptions of specific embodiments of the presentinvention have been presented for the purpose of illustration anddescription. They are not intended to be exhaustive or to limit theinvention to the precise forms disclosed, and obviously manymodifications and variations are possible in light of the aboveteaching. The embodiments were chosen and described in order to bestexplain the principles of the invention and its practical application,to thereby enable others skilled in the art to best utilize theinvention and various embodiments with various modifications as aresuited to the particular use contemplated. It is intended that the scopeof the invention be defined by the claims appended hereto and theirequivalents.

1. A method of executing a set of automated test scripts against aplurality of systems to be tested, said method comprising the steps of:(a) providing an instance of said set of automated test scripts on aserver; (b) writing to a system-under-test web page file hosted on saidserver an initial system-under-test value corresponding to an initialsystem to be tested; (c) retrieving said initial system-under-test valuefrom said system-under-test web page file; (d) launching an initial URLusing said initial system-under-test value retrieved to execute said setof automated test scripts against said initial system to be tested; (e)overwriting, upon completion of execution of said set of automated testscripts against said initial system, said system-under-test web pagefile to change said initial system-under-test value to a nextsystem-under-test value corresponding to a next system to be tested; (f)retrieving said next system-under-test value from said system-under-testweb page file; (g) launching a next URL using said nextsystem-under-test value to execute said set of automated test scriptsagainst said next system to be tested; and (h) repeating steps (e)through (g) until said set of automated test scripts have been executedagainst each of said plurality of systems to be tested.
 2. The methodaccording to claim 1, wherein said writing further comprises: creating aset-system-under-test script for each of said plurality of systems; andexecuting an initial one of said set-system-under-test script created togenerate said system-under-test web page file having said initialsystem-under-test value.
 3. The method according to claim 1, whereinsaid retrieving further comprises: creating a get-system-under-testscript for navigating to said system-under-test web page file on saidserver; adding said get-system script to the beginning of each one of aplurality of test scripts within said set of automated test scripts; andexecuting said get-system-under-test script to retrieve said initialsystem-under-test value from said system-under-test web page file. 4.The method according to claim 1, wherein said launching furthercomprises: converting said initial system-under-test value to saidinitial URL; and navigating a browser to said initial URL to executesaid set of automated test scripts.
 5. The method according to claim 1,wherein said set of automated test scripts are created using a testscript execution engine.
 6. The method according to claim 1, whereinsaid set of automated test scripts are executed using a test scriptexecution manager.
 7. The method according to claim 2, wherein saidoverwriting further comprises: executing another one of saidset-system-under-test script created to change said initialsystem-under-test value to said next system-under-test value on said webpage file.
 8. A method of executing multiple sets of automated testscripts against a plurality of systems to be tested, said methodcomprising the steps of: (a) providing a test script execution managerfor executing said multiple sets of automated test scripts; (b)providing an instance of said multiple sets of automated test scripts ona server; (c) appending a get-system script to the beginning of eachindividual test script within each of said multiple sets of automatedtest scripts for retrieving a desired system value corresponding to adesired one of said plurality of systems to be tested; (d) selecting adesired one of a plurality of set-system scripts corresponding to saiddesired one of said plurality of systems to be tested; (e) selecting oneor more test script sets from said multiple sets of automated testscripts to be executed against said desired one of said plurality ofsystems selected; (f) repeating steps (d) and (e) until at least one setfrom said multiple sets of automated test scripts has been selected foreach of said plurality of systems to be tested; (g) executing saiddesired one of said plurality of set-system scripts for setting saiddesired system value corresponding to said desired one of said pluralityof systems to be tested; (h) executing each of said one or more testscript sets selected against said desired one of said plurality ofsystems to be tested, such that said get-system script is executed eachtime and before the execution of each individual test script within eachof said one or more test script sets selected; and (i) repeating steps(g) and (h) until each of said multiple sets of automated test scriptsselected have been executed against said plurality of systems to betested selected.
 9. The method according to claim 8, wherein saidappending further comprises: creating a get-system script for retrievingfrom a web page file said desired system value corresponding to saiddesired one of said plurality of systems to be tested.
 10. The methodaccording to claim 8, wherein said selecting a desired one of aplurality of set-system scripts further comprises: creating a pluralityof set-system scripts for writing a respective system value to said webpage file hosted on said server, each of said plurality of set-systemscripts corresponding to a respective one of said plurality of systemsto be tested; and adding said desired one of said plurality ofset-system scripts to the beginning of said one or more test script setsselected, such that each of said one or more test script sets selectedis executed against said desired one of said plurality of systems to betested.
 11. The method according to claim 9, wherein said executing eachof said one or more test script sets further comprises: navigating tosaid web page file on said server; reading said desired system valuefrom said web page file; and launching an appropriate URL correspondingto said desired system value to execute each of said one or more testscript sets selected against said desired one of said systems to betested.
 12. A computer readable storage medium having programinstructions to execute a set of automated test scripts against aplurality of systems to be tested, said program instructions performingthe steps of: (a) accessing on a server an instance of said set ofautomated test scripts; (b) writing to a web page file a system valuecorresponding to a desired one of said plurality of systems to betested, said web page file being hosted on said server; (c) retrievingsaid system value from said web page file; (d) launching a URL usingsaid system value to execute said set of automated test scripts againstsaid desired one of said plurality of systems to be tested; (e)overwriting said web page file to change said system value to a nextsystem value corresponding to a next desired one of said plurality ofsystems to be tested; (f) retrieving said next system value from saidweb page file; (g) launching a next URL using said next system value toexecute said set of automated test scripts against said next system tobe tested; and (h) repeating steps (e) through (g) until said set ofautomated test scripts have been executed against each of said pluralityof systems to be tested.
 13. A computer readable storage mediumaccording to claim 12, wherein said writing further comprises: creatingindividual set-system scripts for each of said plurality of systems tobe tested; adding an initial one of said set-system scripts to thebeginning of said set of automated test scripts; and executing saidinitial one of said set-system scripts to generate said web page filehaving said system value corresponding to said desired one of saidplurality of systems to be tested.
 14. A computer readable storagemedium according to claim 12, wherein said retrieving further comprises:creating a get-system script for retrieving from said web page file saidsystem value corresponding to said desired one of said plurality ofsystems to be tested; and adding said get-system script to the beginningof each one of a plurality of test scripts within said set of automatedtest scripts; and executing said get-system script to retrieve saidsystem value from said web page file.
 15. A computer readable storagemedium according to claim 12, wherein said launching further comprises:converting said system value to said URL; and navigating a browser tosaid URL to execute said set of automated test scripts.
 16. A computerreadable storage medium according to claim 13, further comprising:executing another one of said set-system scripts to change said systemvalue to said next system value on said web page file for overwritingsaid web page file.